Japanese Patent No. 3,754,266 discloses a semiconductor device having a trench gate structure and a manufacturing method of the semiconductor device. In the manufacturing method, after a trench is defined, a chemical-vapor deposition (CVD) oxide layer is deposited on the whole surface of a semiconductor substrate to fully fill the trench with the CVD oxide layer. Then, an etching process is performed so that the CVD oxide layer remains only at a bottom portion of the trench, that is, a buried oxide layer (hereafter, referred to as a bottom wall insulating layer) remains at a bottom portion of the trench. Subsequently, ions are implanted to the semiconductor substrate using the bottom wall insulating layer as a mask to form a channel layer along a sidewall of the trench.